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METHOD OF SE6MBNTINQ A THRBB-DIMENSIONAI. DATA SET ALI/>WING USER CORRBCTIOMS 



The invention relates to a method of segmenting a three-dimensional structure 
from a three-dimensional, and in particular medical, data set while making allowance for user 
corrections. The method is performed with the help of a deformable three-dimensional model 
whose surface is formed by a network of mashes that connect the nodes. The invention also 
relates to an image-processing arrangement for performing the method and to a computer 
program for controlling the arrangement. 



A method of the above-mentioned kind, in which however there is, a priori, no 
provision for user corrections, is known from an article by Mclnemey et al. "Deformable 
models in medical image analysis: A survey" in the journal Medical Image Analysis, 1(2): 
91-108, 1996. In this method, a deformable model is represented as an elastic surface that is 
able to change its shape and position under the prompting of what is termed internal and 
external energy. The model having been generated on the basis of pre-existing medical 
knowledge about the structure to be segmented, the function of the internal energy is to 
maintain the shape of the model as weU as possible, whUe the external energy is responsible 
for moving the surface of the model towards the edges of the structure. The external energy is 
derived from the three-dimensional data set. The mashes are, for example, each in the form of 
a triangle whose comers are defined by three adjacent nodes on the surface of the model. 

In the known method, candidate points, i.e. points that are probably situated on 
the surfece of the structure to be segmented, which surface is usually characterized by a 
pronounced gradient shown by the values in the data set, are looked for in the three- 
dimensional data set along normals to the surfeces of the triangles. Once candidate points 
have been determined for the mashes of the network that defines the surface of the model, the 
nodes of the model are re-calculated on the basis of the candidate points that have been 
found. The new nodes are so calculated in this case that the weighted sum of the internal and 
external energies is a minimum for them. The method is flien repeated a number of times on 
the basis of the new nodes calculated, wifli the model becoming a closer approximation of the 
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shape of the structure to be segmented with each repeat. This iterative process comes to an 
end as soon as a discontinuance criterion, e.g. a given number of repeats, is satisfied. 

What is disadvantageous about this and other automatic methods of 
segmentation is that segmentation errors occur, due for example to artifacts in the three- 
5 dimensional data set Such segmentation errors often occur when the distance between a 

structure to be segmented and another structure is small. There are various known methods of 
avoiding segmentation errors, some of which will be explained below. 

A simple method of preventing or correcting segmentation errors is to repeat 
the matching-up process described above with different initial models and with the 
10 deformable model in different initial positions relative to the structure to be segmented. This 
method calls for the result of the matching-up process to be foreseen and hence for a great 
deal of experience on the part of the user. Foreseeing the result in this way is almost 
impossible where the structures to be segmented are complex. Also, the method is very time- 
consuming. 

15 A method of the generic kind on the other hand gives the user an opportunity 

of intervening in the matching-up process, while it is proceeding, by deforming the model 
manually by, for example, displacing one or more nodes. What is disadvantageous about this 
method is user corrections cannot be shown visually in a clear way for the whole of the 
structure to be segmented. The deformable model therefore has to be checked, and if 

20 necessary corrected, layer by layer. Accurate correction is therefore very time-consmning. 

Another known method of the generic kind comprises, after automatic 
segmentation has taken place, deforming the matched-up model manually, e.g. by displacing 
a node. The automatic segmentation is then performed for a second time with this deformed 
model. What is problematic about this method is lhat the step of the method in which the 

25 internal and external energies are nairiiiixized moves the nodes that have been displaced 

manually back to their original positions, because it is at these positions that the energy of the 
deformable model is at a minimum. 

30 It is an object of the present invention to specify a method by which 

segmentation errors can be avoided or corrected at a low cost in time and computing work. 
This object is achieved in accordance with the invention by a method of segmenting a three- 
dimensional structure from a three-dimensional, and in particular medical, data set while 
making allowance for user corrections, which method has the following steps: 



wo 2004/053792 PCT/IB2003/005664 

3 

a) provision of a three-dimensional deformable model whose surface is formed 
by a network having mashes that interconnect nodes at the surface of the model, 

b) positioning of the model at a point in a three-dunensional data set at which the 
structure to be segmented is situated, 

c) manual displacement of nodes, 

d) re-calculation of the nodes of the model with weighted allowance for nodes 
that have been displaced manually. 

In contrast to known methods, in the case of the invention weighted allowance 
is made for the nodes that have been displaced manually when the deformable model is re- 
calculated. This allows each node that has been displaced manually to be allowed for to a 
different degree in the deformation, which allows user correction to take place at only a low 
cost in terms of time. 

A preferred method, in which a greater allowance is made for displaced nodes 
whose distance from a boundary surface of the structure to be segmented is small than for 
displaced nodes whose distance from the said boundary surface is greater, is described in 
Claim 2. 

Claim 3 defines a preferred method of re-calculating the nodes that gives good 
results as far as segmentation is concerned. 

An image-processing arrangement for performing the method according to the 
invention is described in claim 4. Claim 5 defmes a computer program for controlling the 
image-processing arrangement described in claim 4. 

These and other aspects of the invention are apparent from and will be 
elucidated with reference to the embodiments described hereinafter. 



In the drawings: 

Fig. 1 is a diagrammatic view of an image-processing arrangement suitable for 

performing the method according to the invention. 

Fig. 2 is a flow chart of the method according to the invention. 

Fig. 3 shows a deformable model of a dorsal vertebra. 

Fig. 4 shows part of a spinal colunm with a deformable model positioned on a 

dorsal vertebra. 

Fig. 5 shows the deformable model positioned on the dorsal vertebra after a 
known matching-up method has been performed. 
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Fig. 6 shows the defonnable model positioned on the dorsal vertebra after 
manual correction, and 

Fig. 7 shows the deformable model positioned on the dorsal vertebra after the 
method according to the invention has been performed. 



The image-processing arrangement shown in Fig. 1 comprises an image- 
processing and control processor 1, having a memory 2 in which a deformable model of a 
structure to be segmented, and a three-dimensional and in particular medical data set, may be 
stored. The image-processing and control processor 1 may be connected, by a fiber-optic 
cable 3 for example, to an imaging means (not shown) such as a magnetic resonance or 
computer tomography unit. The structure segmented by the image-processing and control 
processor 1 can be shown on a monitor 4. The user can access the image-processing and 
control processor 1 via a keyboard 5 or by means of other input units that are not shown in 
Fig. 1. 

Fig. 2 is a flow chart of a method of segmentation that can be carried out with 
the image-processing arrangement shown in Fig. 1. 

After initialization in step 100, in step 101 the deformable model M of a 
structure to be segmented, and a three-dimensional data set containing the structure to be 

segmented, are loaded. 

Fig. 3 shows a deformable model M that in the present embodiment is a model 
of a dorsal vertebra. Other structures too can be segmented by means of a deformable model, 
e.g. the head of the femur (the structure situated at the top end of tiie thighbone) or an organ, 
such as the heart, for example. The surface of the deformable model M is formed by a 
network whose triangular mashes each connect together tiaree adjoining nodes on the surface 
of the model. Rather than being tiiangulated in this way, the surface of the model could also 
be defined by any desired structure of polygons in which the mashes of tiie network were not 
triangular but connected together a number of nodes other than three to form a polygon. 

A defonnable model can be generated 6om a plurality of training data sets in 
each of vWch the same object has been segmented manually by a skilled operator in such a 
way that each object segmented has the same topology. The taking of a mean, and in the 
simplest case the taking of an arithmetic mean, over all the objects segmented produces an 
object that is referred to as a deformable model. 
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The method according to the invention can also be used where the deformable 
model has been generated in other ways, e.g. by triangulating a model of a dorsal vertebra. A 
method of trianguladon that is frequently employed is known from the article "Marching 
cubes: A high resolution 3d suifece construction algorithm", W.E. Lorensen and H.E. Cline, 
Computer Graphics. 21(3): 163-169, 1987. 

In step 102, the deformable model M is so oriented relative to the structure to 
be segmented that the surface of the model is as close a match as possible to the contours of 

the structure to be segmented. 

In step 103, parts of the deformable model can be altered manually. What a 
user can do m this case is to displace triangular surfeces or individual nodes in the model. In 
step 104. a candidate point jc, is determined for each mask, i.e. in the present embodiment for 
each triangle on the surface of the deformable model. The candidate point is situated on 
the normal m to the surface of a triangle, the normal m to the surface being a unit vector and 
origmating from a center point xi of the triangular surface. The positions Xi of the center 
points define the positions of the relevant triangular surfaces before they were displaced 
manually in step 103, as they may in some cases have been. The position of the candidate 
point on the normal to the surface is now selected in such a way that the function 

/0)=2)/82-F(x,+y5«,) (1) 
is a minimum, hi this function, D is a freely selectable, but constant, value that states how 
much allowance is made for the distance from the surface of the triangle in determining a 
candidate point. Also, d is a scalar distance between two successive points on the normal to 
the surface and j is an index character that designates points on the normal to the surface. The 
so-called edge-detection fimction F(x) is a measure of the probability of point x being 
situated on the surface of the structure to be segmented. This function may be calculated 
from, for example, title following equation: 

m this equation, g(x) is the gradient of the data values (which are brightness values m the 
present embodiment) in the three-dimensional data set at point x. Because the function F(x) 
is the scalar product of the vectors n and g(x), the function becomes particularly large when 
the gradient extends in the direction of the normals to the surfece. What this means is that, if 
it is assumed that the brightness values in the three-dimensional data set change sharply at the 
boundary surface of the structure to be segmented, then the function F(x) is particularly large 
if the normal to the surfece is perpendicular to this boundary surfece. In the present 
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embodiment, it is assumed that the data values for the structure to be segmented are larger in 
the region of the boundary surface than the data values outside the structure. Hence, it is 
assumed that the gradient g(x) points into the structure to be segmented. The normal n to the 
surface on the other hand points outwards. The sign of the fiinction F(x) in equation (2) is 
therefore selected in such a way in the present anbodiment that function F(x) becomes 
positive in the case of a boundary as defined here. In other embodiments, the sign may also 
be so selected that, where the relative sizes of the data values in the region of the boundary 
surface are as mentioned above, the function F(x) is negative. 

The candidate point x, is calculated from the following equation: 

where y, is that integer value of j for which the function defined by equation (1) is a 
minimum. 

Once triangular sxirfaces or nodes have been displaced manually and a 
candidate point has been determined for each triangular surfece on the surface of the 
deformable model, in step 105 the surface of the deformable model is matched to the 
candidate points found and to the manually displaced triangular surfaces or nodes. When this 
is done, the model is deformed in such a way that the so-called energy function 

is a minimum. In this equation, Eext is the external energy that moves the triangular surfaces 
in the direction of the candidate points determined in step 104, whereas Eint is the internal 
energy that acts to oppose any deformation of the network. The energy Euser is an energy that 
causes the model to deform in the direction of the nodes or triangular surfaces that are 
displaced by the user. The weighting factors a and p are typically selected in such a way that 
the three energies contribute in substantially equal proportions to the total energy. 

What may be taken as the external energy is any quantity that becomes smaller 
as the deformable model approaches the structure defined by the candidate points. In the 
present embodiment, the external energy Eexi is defined by the following equation: 

A 

where the summing takes place over all the triangles. The weigihting fector w, is calculated 
from the following equation: 

w, = maK^,F{x,)-Dj,^S^} 
According to equation (6), the weighting factor for candidate points x, that, with a high 
probabiUty, are situated on the boundary surfece of the structure to be segmented, i.e. that 
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have a high F{xi) value, and whose distance from the triangular surfaces is small, is larger 

than for other candidate points. 

The internal energy Eint may be any quantity that becomes all the smaller the 
less the model is deformed. In the present embodiment, internal energy is calculated from the 
equation: 

In this equation, V is the number of triangular surfaces in the model, a- is a scaling factor, R is 
a rotation matrix and Ay is the distance between two center points of two triangular surfaces 
in the original, undeformed model. The set N(i) contains the center points of those triangular 
surfaces that are adjacent to the triangular surface having the center point x,. When energy is 
nunimized for the first time xrnder equation (4), s = 1 and R is the unit matrix. 

The energy £user may be any term that becomes smaller as the deformable 
model is brought closer to the triangular surfaces or nodes that have been displaced, and that 
weights the contributions made by the individual triangular surfaces or nodes that have been 
displaced. In the present embodiment, the energy ^uscr is calculated from the following 
equation: 

A 

where the summing takes place over all the triangles that were altered manually in step 103 
and xi are the center points of these triangles. Multiplication by the weighting factors 
w, = max {O, F(3e, )} causes greater allowance to be made for those triangular surfaces that were 
displaced by the user to a boundary surface of the structure to be segmented than for other 
triangular surfaces. 

In other embodiments, allowance could, in addition, be made, by means of the 
weighting factors w, , for the distance between the center points of the manually displaced 
triangular surfeces and the candidate points. This could, for example, be achieved by 
weighting factors of the following form 

=max^,F(*,)-2)|lx, <9) 
Greater allowance wovdd then be made for manually displaced triangular surfeces that had 
been positioned close to the surface of the structure to be segmented than for triangles that 
had been displaced to positions farther away from the structure to be segmented. 

Once the center points xi of the triangular surfaces of the deformed model have 
been determined by minimiztns the energy in accordance with equation (4), a new scaling 
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factor s and a new rotation matrix R are determined. When this is done, the scaling factor and 
rotation matrix are selected, while allowing for the current center points Xi, in such a way that 
the internal energy becomes a minimum. In other embodiments, the center points Xi, the 
scaling factor s and the rotation matrix R could be determined simultaneously by turning the 
scaling factor and rotation matrix too into variables when determining the total energy E. 

Once the deformable model has been matched to the structure to be segmented 
in step 105, a check is made in step 106 to see whether a discontinuance criterion has been 
satisfied. This discontinuance criterion may, for example, be a predetermined number of 
repeats of steps 103 to 105 or a change in the position of the nodes for which hardly any 
scope remains. If the discontinuance criterion is met, the method of segmentation comes to an 
end at step 1 07. If, on the other hand, the discontinuance criterion has not been met, the 
method continues with step 103. In other embodiments, it could also be continued with step 
104. 

The model obtained by means of the method of segmentation can be stored 
and shown on the monitor. It represents the structure to be segmented that is contained in the 
three-dimensional data set. 

In other embodiments, steps 104 and 105 could be interchanged with step 103. 
What would then be determined first would be tiie candidate points by means of which the 
model would then be deformed with the extemal and intemal energies minimized. In the next 
step, the user would have an opportunity to displace individual triangular surfaces or 
individual nodes of the deformable model. If the subsequent check for satisfaction of the 
discontinuance criterion showed that it had not yet been satisfied, then new candidate points 
would be determined and the model would again be deformed while minimizing the total 

energy, including J^uscr- 

What is more, in other embodiments an additional deformation step 105 may 
be inserted between steps 103 and 104. This allows a check to be made each time on the 
effect the candidate points and the displaced triangular surfaces are having on the 
deformation of the model. 

The benefits of the method according to the invention can be explained by 
reference to Figs. 4 to 7. What can be seen in these Figures is a two-dimensional sagittal slice 
through a three-dimensional data set, the data set having been generated by means of a 
computer tomograph in this embodiment. A deformable model M has been positioned around 
a dorsal vertebra 6 of a spinal column 4. Fig. 4 shows tiiis positioning before a matching 
process is carried out. A typical result of the deformation of the model M by the method 
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mentioned in the introduction can be seen in Fig. 5. In the region of the right-hand section 8 
of the model M a segmentation error has occurred because section 8 of the model also 
contains parts of the adjacent, lower dorsal vertebra 9. In Fig. 6, a manual attempt has been 
made to correct the segmentation error roughly by deforming section 8 of the model, i.e. by 
movmg part of it towards the right. After this rough correction, the edge region 11 of section 
8 of the model is a good match for the boundary surface of the structure to be segmented. 
whei«as the right-hand edge region 13 of section 8 of the model is not such a good match for 
the boundary surface. Fig. 7 shows the model M after the method according to the invention 
has been performed, where the total energy was minimized in accordance with equation (4). 
There has been hardly any change in the left-hand edge region 11. The right-hand edge 
region 13 on the other hand has been more severely deformed and is a good match for the 
boundary surface. The reason for the different treatments given to the manually altered edge 
sections 11, 13 is the weighting factors in equation (8) or (9) as the case may be. The edge 
section 1 1 was situated on or close to the boundary surface of the structure to be segmented, 
which meant that the edge-detection fimction f(3c,) was large, which gave a large weighting 
fector. Edge section 13 on the other hand was a long way away from the boundary surface, 
which meant that f{x,) gave a low value and this resulting in a small weighting factor or 

=0 . The matching of the edge region 13 to the structure to be segmented was thus 
performed solely by minimizing the energy ^ext + aEint- 

By assessing the nodes that have been displaced manuaUy, it becomes 
unnecessary for each node or each triangular surface in the model to be moved as accurately 
as possible to the surface of the structure to be segmented. Therefore, what this assessment 
makes possible is user correction at only a low cost in terms of time. 
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